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I.  INTRODUCTION 

This  report  describes  the  initial  phases  of  a  program  directed 
toward  the  realization  of  complex  adaptive  logical  networks  by  optical  means. 
The  logical  networks  with  which  the  research  is  concerned  are  comprised  of 
threshold  devices  in  various  configurations.  The  general  hypothesis  which 
underlies  the  program  is  that  networks  of  threshold  devices  can  be  designed 
which  will  be  capable  of  useful  application  to  pattern  recognition  problems  of 
several  types,  provided  that  we  can  discover  simple,  inexpensive  means  for 
their  manufacture.  Still  another  notion  which  we  accept  is  that  effective 
solutions  to  such  problems  will  be  found  not  in  terms  of  networks  of  fixed 
behavior,  but  in  terms  of  networks  whose  parameters  can  be  easily  altered 
by  suitable  "action  principles"  or  adaptive  strategies. 

This  point  of  view  is  by  no  means  unique  to  or  original  with  the 
Armour  Research  Foundation,  and  is  perhaps  best  represented  by  the  work 
of  Rosenblatt  on  the  Perceptron.  What  characterizes  our  work  is  the  partic¬ 
ular  interest  in  optical  means  for  the  realization  of  such  networks  and  in  the 
theoretical  problems  associated  with  the  discovery  of  adaptive  strategies 
within  a  frameworks  constrained  by  the  properties  of  the  optical  structures. 

While  the  program  has  been  aimed  at  problems  of  both  a  theoretical 
and  "hardware"  nature,  the  major  effort  during  the  initial  phase  has  been 
directed  at  the  realization  of  a  simple  optical  device  for  adaptive  pattern 
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recognition.  This  report  will  therefore  be  confined  entirely  to  a  discussion 
of  this  aspect  of  the  program  and  will  reference  theoretical  results  only  when 
necessary  to  explain  some  aspect  of  the  design, 

II.  AN  OPTICAL  THRESHOLD  OPERATOR 

We  define  a  picture  P  as  a  two  dimensional  array  consisting  of  N 
finite -area  elements  each  of  which  has  an  optical  transmission  from  the  two¬ 
valued  field  consisting  of  t^  and  t^.  The  values  t^  and  t^  correspond  to  the 
values  0  and  1  in  the  usual  binary  field,  or  the  values  opaque  and  clear  from 
the  subjective  visual  field.  Note  that  if  we  place  such  a  picture  in  the  path  of 
a  uniform,  collimated  beam  of  light,  the  total  flux  passing  through  the  picture 
may  be  computed  as  follows: 


N 

F(P)  =  f  S  t.,  (1) 


where  F  (P)  is  the  total  flux  passing  through  the  picture,  f  the  flux  incident 
on  each  element  of  the  picture,  and  t^  the  transmission  of  the  i  th  picture 
element  according  to  some  arbitrary  labeling  of  the  elements  of  the  set  P. 

If  we  place  a  second  picture  W  in  the  optical  path  and  designate  the  optical 
transmission  of  its  constituent  elements  as  w.,  where  the  w.  are  in  the  range 
0  ^  'w ^  <  1  and  the  i  th  element  of  W  has  incident  upon  it  the  flux  transmitted 
by  the  i  th  element  of  P,  we  may  compute  the  total  flux  passing  through  P  and 
W  as: 


N 

F  (P,  W)  =  f  S  w.  t.  .  (2) 


ARMOUR  RESEARCH  FOUNDATION  OF  ILLINOIS  INSTITUTE  OF  TECHNOLOGY 


-  2 


If  further  we  arrange  to  collect  the  total  flux  passing  through  P 

\ 

and  W  and  cause  it  to  impinge  on  a  flux  sensitive  photodetector ,  we  may 

N 

obtain  an  electrical  signal  proportional  to  S  w.t.  ..  This  electrical  signal 

i  ^  1 

can  be  compared  with  a  threshold  value,  say  ©,  and  a  two  valued  indication, 


N 

S  w.  t.  -  0  . 

-i  '  ^ 


We  note  that 


say  y  established  which  depends  on  the  sign  of 
such  an  apparatus  may  be  interpreted  as  a  threshold  operator  having  a  number 
of  inputs  equal  to  the  number  of  elements  of  resolution  in  the  picture  P  and 
having  weights  confined  to  the  range  of  0<  1.  The  "threshold  operator" 

notion  is  one  of  central  importance  in  much  of  the  work  on  pattern  recognition 
and  is  an  operation,  the  properties  of  which  are  becoming  increasingly  well 
known.  For  the  sake  of  those  who  may  not  be  familiar  with  this  notion, 
however,  or  prefer  a  different  language,  Fig,  1  has  been  included  to  define 
the  threshold  operator  as  a  formal  operator  on  a  binary  array. 

Because  of  the  fact  that  the  power  of  a  threshold  device  having  only 
positive  weighting  values  is  very  much  smaller  than  one  having  both  positive 
and  negative  weights  (in  terms  of  the  number  of  different  functions  of  which 
it  is  capable),  we  should  like  to  remove  the  restriction  on  weighting  values 
which  confine  them  to  the  positive  range.  This  restriction  may  be  removed 
by  either  of  two  techniques  which  may  be  implemented  as  follows.  In  the 
first  implementation,  we  arrange  for  the  weighting  transparency,  W,  to  have 
two  elements  of  resolution  for  each  element  of  resolution  in  the  original 
picture  P,  say  w^^^and  W2£  corresponding  to  the  element  i  in  the  picture  P. 

We  then  employ  some  technique  (for  example,  a  transforming  bundle  of  optical 
fibers)  to  cause  the  flux  from  the  set  of  elements  w^  =  Wj 


12’ 


w 


li’ 


w^^^  to  impinge  on  one  photodetector  and  the  flux  from  the  set 

,  ...jW^-,  ....  to  impinge  on  a  second  photodetector. 


^22 
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If  Ej  is  the  electrical  output  of  the  first  photodetector  and  E2  the  output  of 
the  second  detector  we  see  that  the  quantity  (E^  -  E^)  is  obtained  as  follows; 

N  N  N 

(Ej  -  E2)  =  kf  S  Wj^t.  -  kf  S  ^2^^  =  kf  S(w^.  -  ^ 

where  k  is  proportional  to  the  detector  sensitivity. 

Thus  the  quantity  playing  the  role  of  the  weight  in  the  threshold 
operator  formulation  of  the  above  device  is  device  is  w^  = 

Clearly  w^  can  now  take  on  values  from  the  range  -  Ti  w.  +1. 

Note  that  we  eliminate  the  requirement  for  the  transforming  bundle 
of  optical  fibers  if  we  employ  instead  a  replica  of  P  and  duplicate  systems  to 
obtain  the  quantities  E^  and  E2.  Note  further  that  if  we  add  to  both  copies  of 
the  picture  P  (and  in  fact  to  both  copies  of  every  picture  which  will  be  an 
input  to  the  device)  a  region  in  which  the  transmission  is  always  (for 
example  a  clear  border),  and  designate  the  transmission  of  the  weighting 
transparencies  in  the  corresponding  "border"  areas  as  0^  and  ©2,  then 
the  quantity  E^^  -  E2  will  be  modified  as  follows; 

(El  -  Ei,)=  k 

where  A  refers  to  the  area  of  the  "border"  region  of  P  relative  to  the  area 
of  an  element  of  resolution. 

Making  the  obvious  substitutions  above,  and  adding  a  subscript  j 

to  indicate  the  identity  of  the  jth  picture,  eg,  the  picture  P^  having  elements 

with  transmissions  t. we  see  that 

ij 


N 

S  (w 
i 


li 


Wt)  t.  -  a  t 

Zi'  i 


®l) 


(4) 


N 

Sign 

hi  -  ^2i 

=  Sign 

S  w.  t. .  -  © 
i  ^ 

L  J 
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Having  indicated  how  the  structure  described  above  provides  a 


realization  of  a  threshold  operator  with  optical  input,  we  pose  two  additional 
questions.  First,  "Can  we  elaborate  the  above  structure  so  as  to  provide 
networks  of  threshold  operators  rather  than  single  operators  the  logical  po¬ 
tential  of  which  is  so  extremely  limited?",  and  second,  "Can  we  discover 
convenient  means  whereby  the  structure,  either  in  its  original  form  or  in  an 
elaborated  form,  can  be  rendered  adaptive?".  It  is  primarily  the  second 
question  with  which  this  report  will  deal.  Further  consideration  of  the  elab¬ 
oration  of  the  structure  must  await  developments  in  the  theory  of  such 
adaptive  networks  as  well  as  the  quantitative  results  of  the  effort  to  render 
the  single  operator  adaptive. 

Ill,  ADAPTIVE  TECHNIQUES  FOR  OPTICAL  THRESHOLD  DEVICES 

The  work  of  Rosenblatt,  Joseph,  Widrow,  Stafford,  and  others, 
has  demonstrated  the  fact  that  a  number  of  adaptive  strategies  exist  which 
will  cause  a  single  threshold  device  to  converge  to  any  desired  behavior 
within  its  structural  limitations.  Most  of  the  adaptive  strategies  are  of  the 
error- correcting  variety,  i,e.  a  sequence  of  samples  from  the  set  of  all 
inputs  is  presented  to  the  threshold  device,  the  threshold  device  is  caused 
to  evaluate  each  input  (assign  it  a  name)  based  on  the  existing  weight  and 
threshold  values,  and  the  device  is  modified  if  and  only  if  the  device  response 
(the  assigned  name)  is  not  in  accord  with  the  "desired"  response.  Clearly  in 
this  case  "error"  simply  means  that  the  device  response  to  a  particular  input 
picture  is  not  what  we  should  like  it  to  be.  It  can  be  readily  demonstrated 
that  a  strategy  which  calls  for  a  positive  increment  in  all  weights  whose 
increased  value  would  have  "tended  toward"  the  correct  response,  together 
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with  a  negative  increment  in  those  weights  whose  decrease  will  "tend  toward" 
a  correct  response,  is  an  effective  strategy.  By  "effective  strategy"  we 
mean  one  which  will  result  in  a  finite  number  of  errors  for  any  initial  values 
of  the  weights.  While  the  fact  that  such  a  strategy  will  converge  is  an 
interesting  result,  the  strategy  is  not  one  that  can  be  considered  for  any 
practical  optical  device  since  the  absolute  value  of  the  weights  required  may 
be  extremely  large,  A  more  practical  formulation  of  a  strategy  is  one  in 
which  each  "increase"  takes  the  weight  some  fraction  of  the  way  from  its 
present  value  to  a  "maximum"  value,  and  each  "decrease"  takes  it  some 
fraction  of  the  way  toward  a  "minimum"  value,  eg. 

Aw.  (increase)  =  a 

AWj^  (decrease)  =  a 


w  -  w.  (t) 
max  ’■ 

w  .  -  w.  (t)~1 

min  ^ 


Such  a  strategy  has  also  been  described  as  having  "decay"  since 
we  may  think  of  the  change  in  weight  as  being  comprised  of  two  parts,  a  fixed 
increment  equal  to  +  a  and  a  "decay"  equal  to  aWj(t).  While  such  a  strategy 
can  be  shown  to  converge  in  the  same  sense  that  the  fixed  increment  strategy 
will  converge,  it  will  not  do  so  for  all  functions  if  we  additionally  require 

be  greater  than  some  positive  quantity  for 


S  w.  t..  -  0 
i  1  Ji 


that  the  quantity 

all  j.  Such  a  requirement  corresponds  to  the  demand  that  the  response  be 
unambiguous  since,  if  we  had  no  such  requirement,  we  might  regard  as 


acce^able  a  set  of  weights  which  for  some  inputs  yielded  values  of  the  quan- 
~N 

the  sign  of  which  could  be  altered  by  arbitrarily  small 


tity 


S  w.  t. ,  -  0 
i  1 


changes  in  0.  Let  us  refer  to  the  modified  strategy  as  described  in  equa¬ 


tion  6  as  an  "exponential"  strategy.  The  term  "exponential"  is  used  to  suggest 
the  way  in  which  the  weighting  values  approach  their  limiting  value  as  a  result 
of  a  sequence  of  equal- sign  changes. 
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The  particular  approach  to  an  adaptive  optical  structure  which  is 
being  investigated  at  the  present  time  leads  to  a  strategy  of  the  "exponential" 
variety.  The  problem  of  adaption  in  the  context  of  the  optical  scheme  pre¬ 
viously  discussed  is  that  of  finding  a  scheme  to  generate  the  weighting 
transparencies  appropriate  to  a  specified  behavior  of  the  device.  The 
approach  being  pursued  can  be  most  easily  explained  with  the  aid  of  Fig.  2. 
Figure  2  also  serves  as  a  diagrammatic  representation  of  the  experimental 
apparatus  with  which  this  technique  is  being  evaluated.  Before  attempting  to 
describe  Fig.  2,  however,  we  should  discuss  the  general  properties  of  the 
material  being  utilized  as  the  weighting  transparency.  This  material  is  known 
as  a  photochromic  or  phototropiic  material  and  has  the  property  that  its 
transmission  in  a  portion  of  the  visible  spectrum  may  be  reversibly  altered 
by  exposure  to  relatively  high  intensity  radiation  in  other  portions  of  the 
spectrum.  The  National  Cash  Register  Company  has  been  extremely  active 
in  the  development  of  these  materials  and  has  supplied  the  films  or  trans¬ 
parencies  used  in  the  equipment  to  be  described.  To  be  more  specific 
relative  to  the  properties  of  the  films,  Fig.  3,  curve  A,  shows  the  transmis¬ 
sion  of  a  sample  of  a  particular  film  supplied  by  NCR  in  its  "clear"  state, 
while  curve  B  shows  its  transmission  after  it  has  been  exposed  to  high  inten¬ 
sity  radiation  confined  to  the  blue  end  of  the  spectrum  (in  fact  essentially  a 
white  source  passed  through  the  filter  of  Fig.  4,  curve  B). 

After  subsequent  illumination  with  high  intensity  "yellow"  light, 
eg.  a  white  source  passed  through  the  filter  of  Fig.  4,  curve  Y,  the  transmis¬ 
sion  of  the  photochromic  again  becomes  that  of  Fig.  3,  curve  A.  Thus  we 
see  that  if  we  "read"  the  condition  of  the  photochromic  film  by  means  of  a 
low  intensity  source  operating  over  the  spectrum  defined  by  the  filter  indicated 
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SCHEMATIC  REPRESENTATION  OF  EXPERIMENTAL  APPARATUS 


as  curve  A  in  Fig,  5,  the  observed  transmission  can  be  anywhere  from 
roughly  0.  3  to  roughly  0.  8,  depending  on  the  previous  history  of  the  exposure 
of  the  transparency  to  blue  and/or  yellow  radiation.  It  should  also  be  remarked 
that  although  the  material  is  relatively  stable  (in  the  dark  at  room  temperature 
it  will  decay  toward  an  equilibrium  with  a  time  constant  of  the  order  of  hours), 
we  are  to  some  extent  faced  with  a  kind  of  macroscopic  uncertainty  principle. 

If  we  should  like  to  determine  the  state  of  the  photochromic  by  measuring  its 
transmission,  the  measuring  process  itself  alters  the  transmission  we  seek 
to  measure.  The  saving  feature  is  found  however  in  the  energies  required. 

The  energy  required  to  "read"  the  film  is  so  small  compared  to  that  required 
to  "write"  or  "erase"  the  film  that  the  reading  energy  may  for  all  intents  and 
purposes  be  ignored. 

Returning  now  to  Fig,  2,  we  note  that  the  input  transparency 
(actually  a  35  mm  slide)  is  introduced  into  the  optical  path  from  the  slide 
magazine  of  a  modified  commercial  slide  projector  with  automatic  slide  chang¬ 
ing  features.  The  picture  consists  of  two  side  by  side  versions  of  the  same 
pattern  plus  a  "border"  region.  The  "desired"  response  corresponding  to 
each  input  picture  is  sensed  by  means  of  a  mechanical  switching  arrangement 
actuated  by  the  slide  magazine.  After  a  transparency  has  been  inserted  the 
reading  source,  which  is  placed  at  the  focus  of  the  projection  lens  associated 
with  the  lefthand  projector  of  Fig.  2,  is  turned  on.  The  reading  source  illumin¬ 
ates  the  input  transparency  and  the  photochromic  weighting  transparency  with 
a  uniform  collimated  beam.  The  flux  through  each  of  the  two  halves  of  the 
transparency  is  collected  separately  aid  caused  to  impinge  on  the  two  photo¬ 
detectors.  The  two  detectors  (actually  photoresistors)  are  wired  into  a  bridge 
circuit  so  that  the  direction  of  bridge  unbalance  indicates  the  sign  of 
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FIG.  5  --  READING  FILTER  TRANSMITTANCE 
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is  in  accord  with  the  desired 


N 

N 

S  w.  t.  -  0 

Li"'  _ 

,  If  the  sign  of 

w.t.  -  9 

_i  "  " 

response,  as  indicated  by  the  mechanical  sensing  finger,  the  picture  or 


transparency  is  withdrawn  and  the  next  transparency  inserted.  If  however 
the  response  is  in  "error”,  the  high  energy  gas  discharge  lamp  located  in 
the  right  hand  projector  is  fired  before  the  transparency  is  withdrawn.  The 
right  hand  projector  slide  magazine  contains  two  filter  arrangements,  one 
of  which  is  inserted  in  the  projector  prior  to  the  firing  of  the  high  energy 
source.  The  selection  of  the  filter  actually  inserted  in  each  instance  is  based 
on  the  nature  of  the  "desired"  response.  The  filter  arrangements  each  con¬ 
sist  of  two,  side-by-side  filters,  one  "blue"  (having  the  character  indicated 
in  Fig.  5,  curve  B),  and  one  "yellow"  (having  the  character  indicated  in 
Fig.  5,  curve  Y).  The  two  filters  are  different  in  that  they  are  mirror  images 
of  one  another.  Whichever  filter  arrangement  is  in  the  projector  is  imaged 
by  the  pair  of  projection  lenses  onto  the  photochromic  film  (through  the  input 
transparency)  so  that  the  right-hand  picture  is  flooded  with  blue  light  and 
the  left-hand  picture  with  yellow  light  (or  the  reverse)  whenever  an  "error" 
is  committed  by  the  device.  It  can  be  demonstrated  that  this  structure  im¬ 
plements  an  adaptive  strategy  of  the  "exponential",  error- correcting  variety 
discussed  earlier.  The  device  is  designed  so  that  it  will  automatically  and 
repeatedly  sequence  through  the  set  of  input  pictures  until  it  has  gone  through 
the  entire  set  once  without  error,  at  which  point  it  terminates  the  experiment. 


IV.  EXPERIMENTAL  RESULTS 

Unfortunately  no  experimental  resiilts  of  any  interest  have  as  yet 
been  obtained.  One  of  the  major  problems  associated  with  the  design 
described  above  was  due  to  the  fact  that  the  high  intensity  flash  from  the 
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gas  discharge  lamp  is  also  incident  on  the  photodetectors.  Although  the 
photodetector  output  is  not  significant  at  the  time  at  which  the  flash  lamp  is 
ignited,  all  detectors  investigated  suffered  in  some  way  from  the  exposure 
to  such  high  energy  flashes,  so  that  it  was  found  necessary  to  add  a  shutter 
to  protect  the  detectors.  The  particular  shutter  design  created  still  other 
problems  as  a  result  of  the  vibration  it  introduced  in  the  equipment.  Still 
another  problem  resulted  from  our  inability  to  obtain  photodetectors  which 
were  sufficiently  well  matched  over  the  necessary  dynamic  range  of  incident 
"reading"  illumination.  For  the  above  reasons,  among  others,  a  portion  of 
the  optical  system  has  been  redesigned  to  employ  a  single  detector  and  a 
chopper.  In  this  design  the  single  detector  views  the  right  and  left  half  fields 
alternately.  Also  the  present  chopper  design  eliminates  the  need  for  a 
separate  shutter  to  protect  the  detector  from  the  high  intensity  flash.  It  is 
anticipated  that  the  improved  instrumentation  will  provide  experimental 
results  more  nearly  in  accord  with  the  results  of  the  computer  simulation 
experiments  on  which  the  design  has  been  based. 
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